An instability index theory for quadratic pencils and applications

Primarily motivated by the stability analysis of nonlinear waves in second-order in time Hamiltonian systems, in this paper we develop an instability index theory for quadratic operator pencils acting on a Hilbert space. In an extension of the known theory for linear pencils, explicit connections are made between the number of eigenvalues of a given quadratic operator pencil with positive real parts to spectral information about the individual operators comprising the coefficients of the spectral parameter in the pencil. As an application, we apply the general theory developed here to yield spectral and nonlinear stability/instability results for abstract second-order in time wave equations. More specifically, we consider the problem of the existence and stability of spatially periodic waves for the “good” Boussinesq equation. In the analysis our instability index theory provides an explicit, and somewhat surprising, connection between the stability of a given periodic traveling wave solution of the “good” Boussinesq equation and the stability of the same periodic profile, but with different wavespeed, in the nonlinear dynamics of a related generalized Korteweg-de Vries equation.